Comment about AR spectral estimation Usually an estimate is produced by computing the AR theoretical spectrum at (ˆφ, ˆσ 2 ). With our Monte Carlo

Transcription

1 Comment aout AR spectral estimation Usually an estimate is produced y computing the AR theoretical spectrum at (ˆφ, ˆσ 2 ). With our Monte Carlo simulation approach, for every draw (φ,σ 2 ), we can compute the spectrum and otain a draw for f(ω). Typically the mean of these draws will e similar to the spectrum at (ˆφ, ˆσ 2 ). With this posterior simulation, we have the possiility of computing quantiles, proaility intervals or simply a and for the spectral density. The purpose of the and is to get an idea of the uncertainty of the estimation. 164

2 EEG example. The next figure shows several spectrum curves for 50 draws of (φ,σ 2 ). Recall that the oject phsim has the draws of φ coefficients and sigma2, the draws for the variance of the error term σ

3 50 posterior samples of AR(10) spectrum spectrum frequency 166

4 a=ar(eeg,order=10,aic=f) a$ar=as.vector(apply(phsim,2,mean)) a$var=mean(sigma2) x=spec.ar(a,n.freq=500,plot=f) plot(2*pi*x$freq,x$spec,type="l",axes=f) axis(1) axis(2) for(i in 1:50){ a$ar=as.vector(phsim[i,]) a$var=sigma2[i] x=spec.ar(a,n.freq=500,plot=f) lines(2*pi*x$freq,x$spec) print(i) } 167

5 Portmanteau lack of fit test For this test we need to consider the estimated residuals for the AR model ˆǫ t = x t p j=1 ˆφ j x t j where ˆφ j is some estimator of the model parameters. The purpose of this test is to determine if the residuals are correlated or not.. The null hypothesis is H o : ρ 1 = ρ 2 =... = ρ K = 0 The proposed test statistic is: Q = n(n + 2) K (n k) 1 2 ˆρ k k=1 where ˆρ k is the sample ACF of the estimated residuals and K is a fixed integer. 168

6 The paper y Ljung and Box (1978), On a measure of lack of fit in time series models, Biometrika, 65, shows that under the null hypothesis, Q approximately follows a a chi-square distriution with K (p + 1) degrees of freedom or Q χ 2 K (p+1) The testing procedure is: reject the null hypothesis at the α level if Q > χ 2 K (p+1)(1 α). where χ 2 K (p+1)(1 α) is the (1 α) quantile of the chi-square distriution with K (p + 1) degrees of freedom. A prolem with this test is that there is no formal rule to select the value K. 169

7 A common approach is to compute the p-value of test for different values of K. fit=arima(eeg,order=c(10,0,0)) tsdiag(fit) 170

8 Standardized Residuals Time ACF of Residuals ACF Lag p values for Ljung Box statistic p value lag 171

9 Model order via likelihood approaches: AIC, BIC We want to define a criteria that allows to select the order p of an AR process. We are thinking of the AR model as a linear regression model with p covariates. As p increases the likelihood (or log-likelihood) of the model evaluated at the MLE (ˆφ,s 2 ) also increases. However, as p increases we may have high autocorrelations of regressors. A penalty function could e added to the likelihood function to compensate for more parameters in the model. A general selection criteria is to find the value of p such 172

10 that minimizes 2log[L(ˆφ,s 2 )] + f(p) where L( ) is the likelihood function of the regression model and f( ) is a penalty function. This penalty function f(p) is assumed to e an increasing function of p. Since we are working with a Normal linear model, we can show that 2ln[L(ˆφ,s 2 )] = m(log(2π + 1)) + mlog(s 2 p) where m = n p is the length of the response vector In fact, for the AR model x = Fφ + ǫ, the likelihood 173

11 function L(φ,s 2 ) = ( ) 1 m/2 ( 2πσ 2 exp 1 ) 2σ 2(x Fφ) (x Fφ) Recall that the MLE, ˆφ = (F F) 1 F x and s 2 = (x F ˆφ) (x F ˆφ)/m and so ( ) 1 m/2 ( L(ˆφ,s 2 ) = 2πs 2 exp m ) 2 The first term of 2ln[L(ˆφ,s 2 )] does not depend on p. The criteria reduces to find the value of p for which is minimum. nlog(s 2 p) + f(p) The evaulation must ased on a common sample size. We 174

12 fix a maximum order p and fit AR models for values of p p ased only on n = n p oservations. Then we compute n log(s 2 p) + f(p);p = 0,1,...,p and find the max over the range 0,1,... p If we set f(p) = 2p, we have the Akaike information criteria (AIC). This AIC tends to give overestimated values of p. If we fix f(p) = log(n )p we have the Bayesian information criteria (BIC). The BIC tends to give smaller values of p in comparison to AIC. 175

13 log likelihood a a a a a a a a a a a a a a a a a a AR order 176

14 Forecasting with AR models We will consider forecasting from oth Bayesian and non-bayesian perspectives. We wish to produce inference aout the future. From time n, we wish to produce a statement aout X n+1,x n+2,...,x n+h where h is the forecasting horizon (how far we wish to predict in time). In a Bayesian setup, this translates into considering the Predictive distriution for the future values, p(x n+h, x n+h 1,...,x n+1 x n,...,x 1 ) = p(xn+h, x n+h 1,...,x n+1 x n,...,x 1, φ, σ 2 )p(φ, σ 2 )dφdσ 2 For AR models even with the non-informative prior 177

15 p(φ,σ 2 ) 1/σ 2, this distriution does not have a recognizale form. However, using posterior simulation it is relatively simple to otain samples of values for X n+1,x n+2,...,x n+h We can proceed in the following way: Draw a pair (φ,σ 2 ) from the Normal-Inverse Gamma distriution as we discussed efore. Using this pair, draw a value x n+1 from a Normal distriution with mean p j=1 φ jx n+1 j and variance σ 2. Draw x n+2 from a Normal distriution with mean p j=1 φ jx n+2 j and variance σ 2. (In one of the terms of the autoregression we are using the draw for x n+1 ). 178

16 Continue in this way until we generate a value for x n+h from a Normal with mean p j=1 φ jx n+h j and variance σ 2 Repeat all the steps until we otain M samples of values x n+1,x n+2,...,x n+h An approximation to this scheme is to make draws from a predictive distriution which is conditional to an estimate of the model parameters (ˆφ, ˆσ 2 ) p(x n+h,x n+h 1,...,x n+1 ˆφ, ˆσ 2,x n,x n 1,...,x 2,x 1 ). We are treating (ˆφ, ˆσ 2 ) as the true parameter. If the sample size n is large this should produce similar results with respect to full Bayesian approach that uses 179

17 draws of (φ,σ 2 ). However, if the sample size is small we could find differences etween the distriutions. Once again, consider the EEG data with an AR(10) model. The figures show: Samples of predictive values and data. Comparison of full predictive with MLE predictive Posterior mean of forecasts. Posterior mean and 95% predictive forecasts. Parts of code included in file code6.s 180

18 EEG data and 4 samples from the predictive distriution eeg eeg eeg eeg

19 Sample at MLE compared to sample from predictive eeg eeg

20 EEG data and posterior mean for forecasts eeg time 183

21 EEG time 184

22 # function to produce forecasts # ph are the model coefficients, #h is the forecasting horizon # zt last p values of time series forcar=function(ph,v,h,zt) { x=rep(na,h);p=length(zt) for(i in 1:h) { x[i]=sum(ph*zt)+sqrt(v)*rnorm(1) zt[2:p]=zt[1:(p-1)] zt[1]=x[i] } return(x) 185

23 } p=10 zt=rev(eeg[(n-p+1):n]) forcar(phsim[10,],sigma2[10],200,zt) forcar(phhat,s,200,zt) # 500 samples and mean fr=matrix(na,200,500) for(i in 1:500){ fr[,i]=forcar(phsim[i,],sigma2[i],200,zt) } meanfor=apply(fr,1,mean) 186